Goto

Collaborating Authors

 Bend


SCOP: Evaluating the Comprehension Process of Large Language Models from a Cognitive View

Xiao, Yongjie, Liang, Hongru, Qin, Peixin, Zhang, Yao, Lei, Wenqiang

arXiv.org Artificial Intelligence

Despite the great potential of large language models(LLMs) in machine comprehension, it is still disturbing to fully count on them in real-world scenarios. This is probably because there is no rational explanation for whether the comprehension process of LLMs is aligned with that of experts. In this paper, we propose SCOP to carefully examine how LLMs perform during the comprehension process from a cognitive view. Specifically, it is equipped with a systematical definition of five requisite skills during the comprehension process, a strict framework to construct testing data for these skills, and a detailed analysis of advanced open-sourced and closed-sourced LLMs using the testing data. With SCOP, we find that it is still challenging for LLMs to perform an expert-level comprehension process. Even so, we notice that LLMs share some similarities with experts, e.g., performing better at comprehending local information than global information. Further analysis reveals that LLMs can be somewhat unreliable -- they might reach correct answers through flawed comprehension processes. Based on SCOP, we suggest that one direction for improving LLMs is to focus more on the comprehension process, ensuring all comprehension skills are thoroughly developed during training.


M3DocRAG: Multi-modal Retrieval is What You Need for Multi-page Multi-document Understanding

Cho, Jaemin, Mahata, Debanjan, Irsoy, Ozan, He, Yujie, Bansal, Mohit

arXiv.org Artificial Intelligence

Document visual question answering (DocVQA) pipelines that answer questions from documents have broad applications. Existing methods focus on handling single-page documents with multi-modal language models (MLMs), or rely on text-based retrieval-augmented generation (RAG) that uses text extraction tools such as optical character recognition (OCR). However, there are difficulties in applying these methods in real-world scenarios: (a) questions often require information across different pages or documents, where MLMs cannot handle many long documents; (b) documents often have important information in visual elements such as figures, but text extraction tools ignore them. We introduce M3DocRAG, a novel multi-modal RAG framework that flexibly accommodates various document contexts (closed-domain and open-domain), question hops (single-hop and multi-hop), and evidence modalities (text, chart, figure, etc.). M3DocRAG finds relevant documents and answers questions using a multi-modal retriever and an MLM, so that it can efficiently handle single or many documents while preserving visual information. Since previous DocVQA datasets ask questions in the context of a specific document, we also present M3DocVQA, a new benchmark for evaluating open-domain DocVQA over 3,000+ PDF documents with 40,000+ pages. In three benchmarks (M3DocVQA/MMLongBench-Doc/MP-DocVQA), empirical results show that M3DocRAG with ColPali and Qwen2-VL 7B achieves superior performance than many strong baselines, including state-of-the-art performance in MP-DocVQA. We provide comprehensive analyses of different indexing, MLMs, and retrieval models. Lastly, we qualitatively show that M3DocRAG can successfully handle various scenarios, such as when relevant information exists across multiple pages and when answer evidence only exists in images.


Oregon college student falls to his death after climbing mountain

FOX News

CBP's Air and Marine Operations launched a rescue operation upon request by the Cochise County Sheriff's Office. A college student was located Thursday after he fell several hundred feet while climbing an Oregon mountain. Joel Tranby was climbing North Sister in the Cascade Mountains with his girlfriend early Monday afternoon when he fell about 300 to 500 feet and was severely injured. While Tranby's girlfriend was able to use her phone to call for help, she could not see where Tranby had landed, authorities said. "Unfortunately, he stopped responding verbally before searchers arrived," Lane County Sheriff's Office Sgt.


🇺🇸 Remote Machine learning job: Senior AI Programmer at PlayStation (Bend, Oregon, United States)

#artificialintelligence

Senior AI Programmer at PlayStation United States › Oregon › Bend (Posted May 26 2022) Do they allow remote work? Remote work is possible, see the description below for more information. PlayStation isn't just the Best Place to Play -- it's also the Best Place to Work. Today, we're recognized as a global leader in entertainment producing The PlayStation family of products and services including PlayStation 5, PlayStation 4, PlayStation VR, PlayStation Plus, PlayStation Now, acclaimed PlayStation software titles from PlayStation Studios, and more. PlayStation also strives to create an inclusive environment that empowers employees and embraces diversity.


Is Artificial Intelligence The Future of Avalanche Forecasting? - SnowBrains

#artificialintelligence

Considered by some observers as more of an art than a science, avalanche forecasting involves a unique combination of human observation, analysis, and interpretation. Even the best forecasters only reach about 75% accuracy in their predictions. After all, they are human too and susceptible to bias and imperfection just like the rest of us. One of the biggest challenges in creating an accurate forecast lies in the fact that avalanche danger cannot be precisely measured and is therefore a matter of expert assessment (opinion). That was until last season when the Swiss Institute for Snow and Avalanche Research Group (SLF), a part of the National Swiss Federal Institute for Forest, Snow, and Landscape Research successfully tested a first-of-its-kind, artificial intelligence computer program to assist in its avalanche forecasts.


So … What If Aliens' Quantum Computers Explain Dark Energy?

WIRED

When I lived in the Bay Area, I used to get together with my friend Jaron Lanier to explore the implications of spectacularly weird thought experiments. Outlandish thought experiments have been essential in the intellectual history of science, but the point isn't the weirdness itself. The payoff of thinking about strange things like Schrödinger's cat, the infamous cat that is alive and dead at the same time, is not necessarily that we should then "believe" in the existence of such a cat. Instead, we can hope that uncommon ideas will shed light on the murky margins of our thoughts; in the case of Schrodinger's cat, in dealing with the question of superposition. The point is not to confuse or bamboozle people, but to eventually find a way to think that makes more sense and is a little less murky.


Stretchable distributed fiber-optic sensors

Science

Distributed fiber-optic sensors have been used for monitoring mechanical deformations in stiff infrastructures such as bridges, roads, and buildings, but they either are limited to measuring one variable or require complex optics to measure multiple properties. Bai et al. now demonstrate dual-core elastomeric optical fibers, one of which contains patterned dye regions. The waveguides are fabricated by molding out of commercially available elastomers and integrate a clear core and an adjacent core doped with up to three macroscale dye regions. Changes in optical paths in the two cores detect deformation and map it onto a color space. By monitoring changes in the color and intensity in both elastomer-based fibers, the researchers could distinguish bending, stretching, and localized pressing with a spatial resolution down to ∼1 centimeter. Science , this issue p. [848][1] Silica-based distributed fiber-optic sensor (DFOS) systems have been a powerful tool for sensing strain, pressure, vibration, acceleration, temperature, and humidity in inextensible structures. DFOS systems, however, are incompatible with the large strains associated with soft robotics and stretchable electronics. We develop a sensor composed of parallel assemblies of elastomeric lightguides that incorporate continuum or discrete chromatic patterns. By exploiting a combination of frustrated total internal reflection and absorption, stretchable DFOSs can distinguish and measure the locations, magnitudes, and modes (stretch, bend, or press) of mechanical deformation. We further demonstrate multilocation decoupling and multimodal deformation decoupling through a stretchable DFOS–integrated wireless glove that can reconfigure all types of finger joint movements and external presses simultaneously, with only a single sensor in real time. [1]: /lookup/doi/10.1126/science.aba5504


Beautiful Future: How Deschutes Uses Artificial Intelligence & Machine Learning to Brew Better Beer

#artificialintelligence

Ask any brewer and they'll admit that while beer has likely been around since the dawn of civilization, we're all still learning new ways to brew it more efficiently, creatively, and quickly. But balancing the brewer's art with modern approaches to automation, measurement, and decision making requires brewers to toe a fine line. Take the personality out of the process, and you sacrifice the "craft" in craft beer. Ignore the best tools available, and you waste precious resources that could be better spent on the creative side of the brewing equation. From their outpost on the eastern edge of the Cascades in Bend, Oregon, Deschutes Brewery has tackled this problem in a forward-thinking way, embracing their brew team's passion for tech and programming. Through their operational technology team, they're using a cutting-edge approach to brewing technology aimed at saving time and money, making higher-quality beer, and in turn freeing up company resources for an aggressive innovation program.


2020 Insurance Trends: Good Vibrations for Agile Insurers

#artificialintelligence

If you swim in the waters of insurance and financial services, it's a good time to paddle out and stand up. While the potential blockchain tsunami remains trapped behind legal and regulatory reefs, several good vibrations are creating strong waves to propel you forward. The question for insurers is, how can we get in the best position to ride these new industry dynamics? First, let's look at the line-up of waves to help you launch your insurance growth venture. At PX Venture Studio, we see four exciting trends that are driving change in the insurance market. Let's face it, this is data's world, and we're just living in it.


Amplion's Machine Learning Platform Accelerates Precision Medicine Collaboration

#artificialintelligence

Amplion, a leading precision medicine intelligence company, has released Dx:Revenue, a groundbreaking software solution that enables test providers to identify ideal pharmaceutical partnership opportunities at the right time to advance precision medicine collaboration. Dx: Revenue is an extension of Amplion's core business intelligence platform that leverages proprietary machine learning to deliver tailored insights into pharma and test developer activities. The platform draws from more than 34 million evidence sources such as clinical trials, scientific publications, conference abstracts, FDA cleared and approved tests, lab developed tests, diagnostic and drug pipelines and more in real time, producing prioritized and timely partnering opportunities that are a precise match between a test provider's capabilities and pharma's specific needs. "Precision medicine has a problem," says Chris Capdevila, CEO, Amplion. "There is an insurmountable volume of information with the potential to drive the realization of precision medicine for patients, but accessing that information strategically, effectively and quickly to make the best pharma partnering decisions is beyond human scale. Our company was founded to address this issue by providing critical evidence-based intelligence that supports the strategic decisions pharmaceutical and test developers need to make to be successful."